Skip to content

feat(gen2-migration): generate per-table stacks for multiple DDB tables#14688

Merged
iliapolo merged 22 commits intogen2-migrationfrom
multi-ddb-per-table-stacks
Mar 24, 2026
Merged

feat(gen2-migration): generate per-table stacks for multiple DDB tables#14688
iliapolo merged 22 commits intogen2-migrationfrom
multi-ddb-per-table-stacks

Conversation

@9pace
Copy link
Copy Markdown

@9pace 9pace commented Mar 17, 2026

Summary

  • Generate creates a separate nested stack per DynamoDB table (storage<resourceName>) instead of a single shared storage stack, matching Gen1's per-resource naming and enabling independent refactor/rollback of each table.
  • Refactor threads resourceName through DDB and S3 forward/rollback refactorers for per-table stack lookups.
  • S3 fix: findNestedStack(facade, 'storage') ambiguously matched DDB per-table stacks (e.g. storageactivity). New findS3NestedStack distinguishes the CDK-hashed S3 stack from lowercase DDB stacks.
  • Validation: validateSingleResourcePerStack replaces validateSingleResourcePerCategory, rejecting only when multiple resources target the same stack.
  • Bug fix: resolveOutputs() handles missing Outputs section (CDK omits it when there are no cross-stack references).
  • Discussions app expanded to multi-resource: added S3 avatars bucket and bookmarks DDB table, with test scripts that verify all resources (including direct DDB and S3 SDK tests) at each migration stage. All snapshots and migration-config refreshed.
  • Typecheck fix: added typescript devDependency to backend-only and discussions so npx tsc uses TS 5+ (required for moduleResolution: "bundler" in Gen2 tsconfigs).

E2E Verification

Discussions app: auth, API, 2 DDB tables (activity, bookmarks), S3 (avatars), 2 lambdas. The bookmarks table has no frontend — it exercises the no-Outputs case where CDK omits the Outputs section from the nested stack template.

The avatars frontend:

discussions-app-avatars-scrrenshot
  1. GeneratecreateStack('storageactivity') + createStack('storagebookmarks') + S3
  2. Deploy Gen2 → separate nested stacks: storageactivityA346F297, storagebookmarks210DAF62, storage0EC3F24A
  3. Forward → all resources moved from Gen1 to Gen2
  4. Rollback → all resources moved back to Gen1
  5. Forward again → full round-trip verified
  6. All 389 unit tests pass (48 suites, 67 snapshots)

Test plan

  • Unit tests pass (389/389 in 48 suites)
  • E2E: generate produces per-table stacks for multi-DDB app
  • E2E: deploy creates separate nested stacks (including S3)
  • E2E: forward refactor moves DDB tables and S3 bucket independently
  • E2E: rollback restores original Gen1 state
  • E2E: forward again succeeds (full round-trip)
  • S3 refactorer correctly distinguishes S3 stack from DDB per-table stacks
  • Discussions test scripts verify S3 avatars and bookmarks DDB at each migration stage
  • yarn build and typecheck-migration-apps pass locally

Closes #14608, #14597

9pace added 2 commits January 15, 2026 14:05
Previously, all DynamoDB tables were placed in a single shared storage
stack, causing conflicts when multiple tables existed. Each DDB table
now gets its own nested stack via backend.createStack('storage<name>'),
matching the Gen1 nested stack naming convention.

Also fixes resolveOutputs() crash when a Gen2 storage stack has no
Outputs section (happens when no cross-stack references exist for a
table).

Closes #14608, #14597
@9pace 9pace requested a review from a team as a code owner March 17, 2026 22:29
9pace added 3 commits March 18, 2026 00:10
…ture data

Recapture discussions app snapshots from live deployments:
- Gen1: amplify-discussionsblade-blade-4edfd (activity + bookmarks tables)
- Gen2: amplify-d1skq8aomhb772-e2etest-branch (per-table stacks)

Pre-generate inputs use real amplify-pull data for bookmarks resource.
Pre-refactor templates fetched from deployed CloudFormation stacks.
Post-generate and post-refactor regenerated by test framework.

Also adds bookmarks to migration-config.json and fixes snapshot.ts
to use TemplateStage: Original for unprocessed templates.
Replace raw app ID d1skq8aomhb772 with sanitized name 'discussions'
in all snapshot file contents and filenames.
9pace and others added 4 commits March 20, 2026 14:46
…er-table-stacks

# Conflicts:
#	amplify-migration-apps/discussions/README.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Re-deployed discussions app with 3 storage types (activity DDB,
bookmarks DDB, avatars S3) and captured fresh snapshots for all 4
stages. Per-table stacks refactor verified working for both DDB tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The S3 forward/rollback refactorers used findNestedStack with a bare
'storage' prefix, which matched DDB per-table stacks (e.g.
'storageactivity') before the S3 stack ('storage0EC3F24A'). Fixed by
using 'storage' + resourceName for Gen1 lookup and a new
findS3NestedStack method that distinguishes the CDK-hashed S3 stack
from lowercase-prefixed DDB stacks for Gen2 lookup.

Also refreshes discussions snapshots from a clean main/gen2-main deploy.
Comment thread packages/amplify-cli/src/commands/gen2-migration/cfn-template.ts
9pace added 5 commits March 23, 2026 20:02
- Replace fragile char-based findS3NestedStack with template-based S3
  detection (fetch template, check for AWS::S3::Bucket resource)
- Export findS3NestedStack from storage-forward.ts, import in rollback
- Remove dead STORAGE_RESOURCE_TYPES export
- Remove storage:S3 from validateSingleResourcePerStack (only one
  defineStorage() possible)
- Make scopeVarName required (no default) in DynamoDBRenderer
- Set Outputs default empty object in cfn-output-resolver, use
  templateOutputs consistently
- Add comments for optional Outputs in cfn-template.ts and resolver
Keep S3 validation as a defensive spot check even though the collision
can't happen in practice (JSON key uniqueness prevents same-name
resources within a category).
Resolved conflicts:
- refactor.ts: pass both logger and resourceName to storage constructors
- category-plan-orchestration.test.ts: use noOpLogger() + resourceName,
  adopt upstream error messages (throws on missing stacks)
- refactor.test.ts: use step.forward() with per-stack validation message
… script

Extends the discussions gen1 test script to cover all stateful resources
that get refactored during migration: S3 avatars bucket (upload/getUrl/remove)
and bookmarks DynamoDB table (put/get/delete via AWS SDK). Verified with
full forward→rollback→forward round-trip on live app.
…sed table lookup

Remove ListTablesCommand discovery in favor of reading table name from
amplifyconfiguration.json. Share a single DynamoDBDocumentClient across
all bookmark test functions instead of creating one per call.
@9pace 9pace changed the title feat(cli): generate per-table stacks for multiple DDB tables feat(gen2-migration): generate per-table stacks for multiple DDB tables Mar 23, 2026
Remove .gitignore that ignored local dev helper scripts
(add-bookmarks.ts, deploy-gen1.ts) that no longer exist.
…solution

backend-only and discussions were missing typescript as a
devDependency, causing npx tsc to fall back to the root
tsc v4.9.5 which doesn't support moduleResolution:"bundler".
Also restores monorepo metadata in discussions snapshot
package.json files.
@9pace 9pace force-pushed the multi-ddb-per-table-stacks branch from a91fce3 to a514ec7 Compare March 24, 2026 01:12
9pace added 5 commits March 24, 2026 10:14
…pshots

Replace "Your verification code is {####}" with "Here is your verification code
{####}" to match the discussions README instructions, preventing drift when
snapshots are regenerated.
Only email verification fields should use the README-specified message.
SMS fields should retain their original values.
Comment thread amplify-migration-apps/backend-only/package.json Outdated
@iliapolo iliapolo enabled auto-merge (squash) March 24, 2026 19:36
@iliapolo iliapolo merged commit 016b5af into gen2-migration Mar 24, 2026
4 checks passed
@iliapolo iliapolo deleted the multi-ddb-per-table-stacks branch March 24, 2026 19:37
iankhou pushed a commit that referenced this pull request Mar 26, 2026
…es (#14688)

* feat: avatar s3 feature for discussions app

* feat(cli): generate per-table stacks for multiple DDB tables

Previously, all DynamoDB tables were placed in a single shared storage
stack, causing conflicts when multiple tables existed. Each DDB table
now gets its own nested stack via backend.createStack('storage<name>'),
matching the Gen1 nested stack naming convention.

Also fixes resolveOutputs() crash when a Gen2 storage stack has no
Outputs section (happens when no cross-stack references exist for a
table).

Closes #14608, #14597

* fix(discussions): replace hand-crafted snapshots with real infrastructure data

Recapture discussions app snapshots from live deployments:
- Gen1: amplify-discussionsblade-blade-4edfd (activity + bookmarks tables)
- Gen2: amplify-d1skq8aomhb772-e2etest-branch (per-table stacks)

Pre-generate inputs use real amplify-pull data for bookmarks resource.
Pre-refactor templates fetched from deployed CloudFormation stacks.
Post-generate and post-refactor regenerated by test framework.

Also adds bookmarks to migration-config.json and fixes snapshot.ts
to use TemplateStage: Original for unprocessed templates.

* docs(discussions): add bookmarks table setup instructions to README

* fix(discussions): sanitize Amplify App ID in Gen2 snapshot files

Replace raw app ID d1skq8aomhb772 with sanitized name 'discussions'
in all snapshot file contents and filenames.

* docs(discussions): add S3 avatars bucket to migration config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(discussions): update snapshots with S3 avatars + bookmarks DDB

Re-deployed discussions app with 3 storage types (activity DDB,
bookmarks DDB, avatars S3) and captured fresh snapshots for all 4
stages. Per-table stacks refactor verified working for both DDB tables.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): disambiguate S3 nested stack from DDB per-table stacks

The S3 forward/rollback refactorers used findNestedStack with a bare
'storage' prefix, which matched DDB per-table stacks (e.g.
'storageactivity') before the S3 stack ('storage0EC3F24A'). Fixed by
using 'storage' + resourceName for Gen1 lookup and a new
findS3NestedStack method that distinguishes the CDK-hashed S3 stack
from lowercase-prefixed DDB stacks for Gen2 lookup.

Also refreshes discussions snapshots from a clean main/gen2-main deploy.

* fix(cli): address PR review comments for per-table stacks

- Replace fragile char-based findS3NestedStack with template-based S3
  detection (fetch template, check for AWS::S3::Bucket resource)
- Export findS3NestedStack from storage-forward.ts, import in rollback
- Remove dead STORAGE_RESOURCE_TYPES export
- Remove storage:S3 from validateSingleResourcePerStack (only one
  defineStorage() possible)
- Make scopeVarName required (no default) in DynamoDBRenderer
- Set Outputs default empty object in cfn-output-resolver, use
  templateOutputs consistently
- Add comments for optional Outputs in cfn-template.ts and resolver

* fix(cli): restore storage:S3 case in validateSingleResourcePerStack

Keep S3 validation as a defensive spot check even though the collision
can't happen in practice (JSON key uniqueness prevents same-name
resources within a category).

* feat(cli): add S3 avatars and bookmarks DDB tests to discussions test script

Extends the discussions gen1 test script to cover all stateful resources
that get refactored during migration: S3 avatars bucket (upload/getUrl/remove)
and bookmarks DynamoDB table (put/get/delete via AWS SDK). Verified with
full forward→rollback→forward round-trip on live app.

* refactor(cli): clean up bookmarks test - shared DDB client, config-based table lookup

Remove ListTablesCommand discovery in favor of reading table name from
amplifyconfiguration.json. Share a single DynamoDBDocumentClient across
all bookmark test functions instead of creating one per call.

* chore(cli): remove dev-only .gitignore from amplify-cli package

Remove .gitignore that ignored local dev helper scripts
(add-bookmarks.ts, deploy-gen1.ts) that no longer exist.

* fix(cli): add typescript devDep to fix typecheck for bundler moduleResolution

backend-only and discussions were missing typescript as a
devDependency, causing npx tsc to fall back to the root
tsc v4.9.5 which doesn't support moduleResolution:"bundler".
Also restores monorepo metadata in discussions snapshot
package.json files.

* fix: use README-specified emailVerificationMessage in discussions snapshots

Replace "Your verification code is {####}" with "Here is your verification code
{####}" to match the discussions README instructions, preventing drift when
snapshots are regenerated.

* fix: replace remaining old verification message in discussions snapshots

* fix: align gen2 branch verification messages with README

* fix: revert SMS verification messages to original values

Only email verification fields should use the README-specified message.
SMS fields should retain their original values.

* fix: remove trailing newlines from parameters files

* fix: remove unnecessary typescript devDependency from migration apps

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants